(navigation image)
Home American Libraries | Canadian Libraries | Universal Library | Community Texts | Project Gutenberg | Children's Library | Biodiversity Heritage Library | Additional Collections
Search: Advanced Search
Anonymous User (login or join us)
Upload
See other formats

Full text of "Hacks and Mods"

Air Guitar Hero 



.[ 



Make Projects 



build, hack, tweak, share, discover. 



Air Guitar Hero 



Written By: Carol Reiley 



TOOLS: 



Matlabd) 

or newer, from MathWorks (http://mathworks.com) 

Matlab Data Acquisition Toolbox (1) 
from MathWorks 

Matlab Signal Processing Toolbox (1) 
from MathWorks 

MiniVIEm 

from the My open project 

(http://code. google, com/p/my open/source/checkout) 

Rotary Tool (1) 
e.g. Dremel 

Screwdrivers (2) 

Soldering iron (1) 



PARTS: 



Nintendo Wii video game 
console (1) 

Guitar Hero III: Legends of 
Rock video game. Wireless 
Bundle with Gibson Les Paul 
controller (1) 

DB9 D-sub connector, 
female, solder type (1) 
#B000l-21YX2at 
amazon.com 

RS-232 serial extension 
cable. dB9 male-to-female (1) 
item #0271 1 from Cables To 
Go (cablestogo.com) 

Laptop computer running 
Windows XP (32-bit version) 

m 

EMG electrodes, silver-silver 
chloride, bipolar (1 box) (1) 
We use Duotrodes. item 
#6145 from Myotronics 
(myotronics.com). These are 
consumables each time you 



© Make Projects 



www.makeprojects.com 



Page 1 of 19 



Air Guitar Hero 



play; a box contains 135. 

Button snap-connectors (1) 
from any fabric store 

Myoelectrode amplifiers (4-8) 
One per electrode pair. The 
more electrodes you use, the 
more accurately Air Guitar 
Hero can decode your finger 
motions. 

Data acquisition board (1) 
item #USB-1208FS from 
Measurement Computing 
(http://mccdaq.com). Used 
for data logging and analysis, 
it samples real -world signals 
(e.g.. force. EMG) and sends 
them to a computer for 
processing. 

USB video capture device (1) 
We used a generic device, 
model number VC-21 1A. but 
any one that supports 
640x480 or 720x480 NTSC 
should work. 



SUMMARY 

By Robert Armiger and Carol Reiley 

We created Air Guitar Hero as a fun rehabilitation exercise for people with amputations. 
Here we'll show you how to make an inexpensive version so anyone can play Guitar Hero 
without pushing buttons. It uses an electrode cuff, a modified Wii guitar controller, and open 
source code. 

Wii-Hab Lab: How the Air Guitar Hero system works. When a muscle contracts or flexes, it 
produces electrical activity. While faint (in the millivolt range), these signals can be detected 



© Make Projects 



www.makeprojects.com 



Page 2 of 19 



Air Guitar Hero 

by placing electrode sensors on the skin. The technology to measure, evaluate, and process 
muscular electricity is called electromyography (EMG). 

Air Guitar Hero uses EMG to send signals to the Wii console to control the game. But since 
the electrical signal generated by twiddling your fingers is very weak, additional computation 
must be performed to generate reliably accurate commands. The system uses pattern 
recognition algorithms to identify patterns in the EMG signals and decide which colored 
button to activate. 

The algorithms require training data to provide examples of what signal characteristics to 
look for. First, you must correctly play on-screen notes with the guitar while the electrodes 
record your EMG signals. 

Next, the recorded data is used to train a model for recognition the next time you make 
those movement patterns. 

Third, practice makes perfect! Playing this type of video game can be useful for building 
muscle tone and dexterity. 



© Make Projects www.makeprojects.com Page 3 of 1 9 



Air Guitar Hero 



Step 1 — Modify the guitar controller. 






Our system needs to send computer-generated commands to the game console. We do 
this by sending digital output commands from the USB-1208FS data acquisition board. Our 
quick-and-dirty approach is to modify the guitar controller and override the manual buttons 
you'd normally press. 

Make sure your Wii Remote is disconnected from the guitar controller. Unlock and remove 
the guitar neck and the front faceplate. Remove the four #0 screws on the front of the 
guitar and the eight T-10 Torx screws on the back of the guitar to open the controller. 
(Don't forget the gratifying removal of the "Void Warranty" sticker to get to the last Torx 
screw). 

Using a Dremel tool, bore out a small D-shaped hole to fit the DB9 solder pot connector. 
Score along the outline of the connector, incrementally removing material and checking fit 
with the connector. Be sure to leave enough material on the sides of the connector for the 
mounting screws. 

Once the connector fits, score the center location of the mounting screws, drill tiny pilot 
holes, and insert 2 small screws (or use 2 of the 8 Torx screws you removed), ensuring 
that everything fits. 

Remove the solder pot connector for wiring. Add seven 12" leads to solder pot pins 1-7 
(one ground, plus 5 note buttons, plus the Strum button). Connect pins 1-6 to the 6 
through-hole pads at the base of the guitar neck, from right to left if the push pins are on 
the bottom. Connect pin 7 to one of the Strum switch contacts; either Up or Down will 
suffice. Ensure all wires are secure and close up the guitar. 

Check your work! Use a multimeter to perform a continuity check between pin 1 (Ground) 
and each of pins 2-7 (Green, Red, Yellow, Blue, and Orange buttons, and Strum 
Up/Down). There should be no connection until each corresponding button is pressed. 



© Make Projects 



www.makeprojects.com 



Page 4 of 19 



Air Guitar Hero 



Step 2 — Configure the data acquisition board. 




Next, use a 3' section of serial 
cable and wire it to the digital I/O 
pins of the USB-1208FS data 
acquisition board. You only need 
half of the 6' serial cable, so cut off 
the male-pin half and use it to wire 
to the data acquisition board. 

Connect pin 1 of the serial cable to 
terminal 29 (GND) of the data 
acquisition board. Connect pins 2- 
6 of the cable to digital ports. 

A0-A4 (terminals 21-25) 
respectively, and pin 7 of the cable 
to digital port A7 (terminal 28). 



© Make Projects 



www.makeprojects.com 



Page 5 of 19 



Air Guitar Hero 



Step 3 — Verify computer control of the game. 



© Make Projects www.makeprojects.com Page 6 of 1 9 



Air Guitar Hero 



-IDIXJ 



SSt Help 



^™ Jp» nf JFh 



A/D D/fl 



fi 



ff_ PC Board List 

-; -^B^ Universal Serial Bus 

Rl BD*d#Q-U5B-1208F5 (serial* 198) 







*J 




U5B-1208FS | 




Factor Serial Na 11 C59F3 




It 1.00 




Custom Seiial No |l93 












No. of Channel Klsrmj; 


Md 




E«t. Clock Type: | Contniuous 


d 




Flash LED 


1 






Ffcnwjaie Versit 




L 


DK 


Cann 









" ! -" 



At this point, a good intermediate 
checkpoint is to connect the data 
acquisition device and test Guitar 
Hero in Auto-Play mode. By routing 
the video signal to your PC, Matlab 
can detect each oncoming note 
from the game and then 
automatically send the digital signal 
to play that note for you. The 
software looks for the bright pixel 
intensities above the fret- board 
within Guitar Hero and then plays 
the corresponding note. (This is 
especially useful if there's a song 
in Expert mode that you just can't 
beat!) 

Connect the USB video capture 
device to the Wii and to your 
computer, and install the drivers on 
your computer. This will allow the 
game to run on your PC. 

Optional: Download and install 
Audacity 

( http://audacity.sourceforge.net ) to 
allow "Software Play-through" of 
the game's audio channels through 
your laptop speakers. 

Attach the USB-1208FS to your 
computer and install the drivers 
from http://makezine.com/go/usb- 
data. Once the drivers are 
installed, open the configuration 
utility (Start -> All Programs -> 
Measurement Computing -^ 
InstaCal), and ensure that the 
device appears in the list of 
detected PC boards and that it's 



© Make Projects 



www.makeprojects.com 



Page 7 of 19 



Air Guitar Hero 



configured for 8 single-ended 
analog inputs (as opposed to 4 
differential inputs). 

Download the Mini VIE package 
from the Subversion repository at 
http://makezine.com/go/minivie . 
This is a package library based on 
the Johns Hopkins University 
Applied Physics Laboratory Virtual 
Integration Environment (JHU/APL 
VIE). It has a basic signal 
simulator that you'll use to test the 
functionality of your modified guitar 
controller. 

Open Matlab and navigate to the 
MiniVIE directory. Type the 
following commands at the Matlab 
prompt: 

• >>MiniVIE. conf igurePath 

>>Presentation . AirGuitarHero .A 
GH 



Step 4 



■ mjHJMIBH 



Jnljsl 



File £drt yew lwert Iwte desktop w*>dm Help 




The Wii console will display on 
your PC within Matlab. Use the 
guitar controller to select and start 
the game. Once the song begins, 
click Auto-Play. The MiniVIE will 
graphically detect notes and then 
play them using the digital output 
pins automatically! 



© Make Projects 



www.makeprojects.com 



Page 8 of 19 



Air Guitar Hero 



Step 5 — Train the pattern recognition system. 



MT^^T^^B 




00C 1000 120Q 1400 1600 1800 2000 

Time (s) 



.jaljsl 



TF, 



1 III s 



I la 11 u n 



Before you hook up electrodes and 
do anything with the EMG signal, 
it's worthwhile to get a feel for how 
the system operates. This can be 
done with the EMG Signal 
Simulator that's built into the 
MiniVIE system. You'll use the 
simulator to walk through each step 
of acquiring, processing, 
classifying, and displaying signal 
outputs. 

Start the MiniVIE by typing in 
Matlab: 

• >>MiniVIE 

In the user interface that appears, 
select the EMG Simulator from the 
Inputs drop-down dialog. A small 
window will pop up in the bottom 
left side of the screen. When the 
window is selected, typing 
keyboard keys a , s, d, f , etc., 
will generate simulated signals as if 
a real DAQ system was present 
and connected to electrodes. 

Preview the signal patterns by 
selecting the SignalViewer button. 
Each time a valid key (a, s, d, 
f ) is pressed in the Signal 
Simulator window, a new pattern of 
simulated EMG activity will appear 
on the screen. Typically one of the 
selected channels will get "noisier" 
in both frequency and amplitude for 
a selected pattern. 



© Make Projects 



www.makeprojects.com 



Page 9 of 19 



Air Guitar Hero 



© Make Projects www.makeprojects.com Page 10 of 19 



Air Guitar Hero 



Step 6 



© Make Projects www.makeprojects.com Page 11 of 19 



Air Guitar Hero 




Next, choose linear discriminant 
analysis (LDA) as the pattern 
recognition algorithm to use for 
decoding signals by selecting LDA 
Classifier from the Signal Analysis 
drop-down menu. The classifier will 
be used in the following steps to 
attempt to recognize patterns from 
the input signals and label them as 
one of several discrete classes. 
Click on Select Channels and 
select Index, Middle, Ring, Little, 
and No Movement as the 
classifier's available choices. 

To use this pattern classifier, you 
need to create a training data set. 
From the Training drop-down menu, 
select Simple Trainer. Click the 
Begin Training button and a series 
of prompts will begin asking you to 
perform each of the movements 
you selected for the classifier (i.e. 
Index, Middle, Ring, Little). Each 
prompt begins by displaying the 
current class as text while a red 
bar moves from left to right. 

When the bar turns green (or 
slightly before), recording begins 
and you should perform the 
requested motion. In this case, 
while the green bar indicates data 
is recording, press the f key with 
your index finger. When the next 
prompt appears for Middle, press 
the d key, then s during Ring, and 
finally a during Little. In each case 
you're indicating to the software to 



© Make Projects 



www.makeprojects.com 



Page 12 of 19 



Air Guitar Hero 



generate the EMG pattern that will 
appear once you attach real 
electrodes. 

After completion, you can save 
your training data in the dialog that 
appears. Resulting accuracy is 
displayed in the console window. 
For example: 

• Training LDA with 551 
Samples (1=105; 2=109; 
3=111; 4=112; 5=114;) 
Percent correctly 
classified: 100.0% Index 
Class accuracy: 100.0% 
Middle Class accuracy: 
100.0% Ring Class 
accuracy: 100.0% Little 
Class accuracy: 100.0% 
No Movement Class 
accuracy: 100.0% 

Once the training set has been 
created and the pattern recognition 
algorithm has been trained with 
output data, you can visualize 
these effects in a number of ways. 
From the Presentation drop-down 
menu, select the MiniV 
visualization utility. Pressing the 
keys a, s, d, and f will cause 
the system to recognize simulated 
finger contraction patterns and 
display the corresponding finger 
movement in the visualization. 



© Make Projects www.makeprojects.com Page 13 of 19 



Air Guitar Hero 



Step 7 — Acquire EMG signals from finger movement. 



© Make Projects www.makeprojects.com Page 14 of 19 



Air Guitar Hero 




Air Guitar Hero processes 
electromyogram (EMG) signals to 
decode your finger movements. To 
acquire these signals, you'll 
connect the input side of the 
electrodes to your arm via stick-on 
electrodes and a snap connector. 

The EMG signal is a high- 
frequency white noise signal of 
several millivolts, with the most 
frequency energy in the 100Hz- 
400Hz range. The amplifiers act as 
an instrumentation (differential) 
amplifier with a variable gain 
(approximately 1,000x), which 
rejects common-mode signal (e.g., 
the 60Hz line noise coupled into 
our bodies) and amplifies local 
biopotential differences caused by 
muscle contractions (i.e., the EMG 
signal). 

The USB-1208FS data acquisition 
system has a +5V power 
connection that can be used to 
power the electrode amplifiers. 

The Otto Bock electrode amplifiers 
have 2 wires plus a shield on the 
input side (skin- side, unamplified), 
and 3 wires on the output side 
(amplified): +5V (power), ground, 
and signal. The output signal is 
centered at 1.2V. 

If you build the electocardiogram 
(ECG) circuit from the schematic in 
Medical Instrumentation: 
Application and Design, note that 
its voltage and frequency ranges 



© Make Projects 



www.makeprojects.com 



Page 15 of 19 



Air Guitar Hero 



are slightly different than EMG 
amplifiers. EMG amplifiers have 
wider frequency response than 
ECG amplifiers, but do not cover 
so low a frequency range. The 
EMG amplifier must have higher 
gain than the ECG amplifier for the 
same output-signal range. 

• Disconnect your laptop from the 
wall supply. Although there's 
minimal risk here, with electrodes 
attached directly to your skin it's 
best to take every precaution to 
ensure you're isolated from any 
power fault your building might 
experience. This also helps to 
ensure that line noise isn't coupled 
into the data acquisition board. 

• Solder 2 snap-connectors to the 
input end of each electrode 
amplifier. Additionally, solder a 
third female snap-connector to the 
ground shield of one of the 
electrode amplifiers. 



© Make Projects www.makeprojects.com Page 16 of 19 



Air Guitar Hero 



Step 8 



H itif i ! *T«*<j-wi j _.>*(.L.-.» J i iy r n 



200 400 600 800: 1000 1200 1400 1600 1800 2000 

Tinae (s) 





,..., 




FM ftoperties 


Pntum 




T« 


» l 





^i*l 



rrrrFiJ i 1 1 1 ' I » I 



Connect all the electrode amplifier 
output ground wires to the data 
acquisition board's digital ground 
(terminal 31) and all the electrode 
power wires to the board's +5V 
(terminal 30). Connect electrode 
signal outputs indi- vidually to 
analog input channels CH0-CH3. 

Apply stick-on electrodes equally 
spaced around your forearm, about 
1" below your elbow. Place the 
stick-on ground somewhere in the 
middle of your arm about 1" away 
from other electrode contacts. 
Snap on each amplifier lead wire to 
the electrodes. 

Start the MiniVIE program again by 
entering >> MiniviE at the 
Matlab com- mand prompt. In the 
pop-up GUI, select DaqHwDevice 
from the drop-down menu, then 
press the Signal Viewer button to 
preview and display the signals. 
Verify that the signals displayed 
correspond to the contraction of 
your forearm muscles by moving 
your wrist and pinching each of 
your Index, Middle, Ring, and Little 
fingers. 



© Make Projects 



www.makeprojects.com 



Page 17 of 19 



Air Guitar Hero 



Step 9 — Rock out! 




Once you've tested the individual 
components, it's time to put it all 
together to check out your Air 
Guitar Hero chops. 

Disconnect your laptop from the 
wall supply. Ensure that the EMG 
electrodes are well attached to 
your forearm (too much electrode 
gel on the adhesive can make them 
fall off). 

Open Mini VIE and select your input 
hardware device. Use the Signal 
Viewer to verify that the muscle 
signals are at rest when you're at 
rest; you should only see EMG 
activity when you're actively 
pinching your fingers. 

Select the LDA classifier and 
choose Index, Middle, Ring, and No 
Movement, initially for beginner 
mode. Train each of these classes 
using the Simple Trainer interface. 
Verify that your accuracy is at 
least 80% or higher. 

Start your Wii console and select a 
song to play using the guitar 
controller, then select AGH from 
the Presentation drop-down and run 
it. In the default configuration, you 
don't have to use the Strum button. 
Just press your fingers when it's 
time to play the note, and it will be 
played with a strum added. 



© Make Projects 



www.makeprojects.com 



Page 18 of 19 



Air Guitar Hero 



Step 10 




That's it! Now you can play Guitar 
Hero simply by making muscle 
contractions and thinking about 
playing the desired note. 

And after you become a Rock God, 
you can modify your Wiimote — 
see our WiiEMG hack at 
http://makezine.com/go/wiiemg — 
to play games like Wii Sports 
Tennis or Mario Kart! 



We'd like to thank Jonathan Kuniholm, founder of http://openprosthetics.org and Duke University 
doctoral candidate, and Jacob Vogelstein of the Johns Hopkins University Applied Physics 
Laboratory for their help creating and testing the first Air Guitar Hero system back in 2007. 

This project first appeared in MAKE Volume 29 . pages 44-51. 

This document was last generated on 2012-11-02 11:54:50 PM. 



© Make Projects 



www.makeprojects.com 



Page 19 of 19